Visual Foresight Trees for Object Retrieval From Clutter With Nonprehensile Rearrangement

This paper considers the problem of retrieving an object from many tightly packed objects using a combination robotic pushing and grasping actions. Object retrieval in dense clutter is important skill for robots to operate households everyday environments effectively. The proposed solution, Visual Foresight Trees (VFT), intelligently rearranges surrounding target so that it can be grasped easily. Rearrangement with nested nonprehensile actions challenging as requires predicting complex interactions combinatorially large configuration space multiple objects. We first show deep neural network trained accurately predict poses when robot pushes one them. predictive provides visual foresight used tree search state transition function scene images. returns sequence consecutive push yielding best arrangement object. Experiments simulation real approach outperforms model-free techniques well model-based myopic methods both terms success rates number executed actions, on several tasks. A video introducing VFT, experiments, accessible at https://youtu.be/7cL-hmgvyec. full source code available https://github.com/arc-l/vft.